REMARKS 



Claims 1-33 are pending in the application. Claims 1, 3, 4, 1 1, 12, 14-20, 
24 and 29-33 have been amended. Reconsideration of this application is 
respectfully requested. 

It is noted with appreciation that the Office Action has indicated that claims 
11, 19 and 23 would be allowable if rewritten to include all the limitations of the 
base claim and of any intervening claims. 

Claims 1 1 and 19 have been amended for antecedent compatibility with 
their respective claims base claims 1 and 18, which have also been amended. It 
is submitted that these antecedent amendments do not affect the allowability 
status of claims 1 1 and 19. 

The Office Action rejects claims 1-3, 5, 6, 9, 18, 24, 31 and 33 under 35 
U.S.C 103(a) as unpatentable over U.S. Patent No. 5,500,920 to Kupiec, 
hereafter Kupiec in view of U.S Patent No. 6,581,054 to Bogrett, hereafter 
Bogrett. 

Independent claims 1, 18, 24 and 33 have been amended to recite that 
the events are derived from the information stream based on one or more 
predetermined taxonomies. Similar taxonomy language has previously been and 
continues to be in claims 3, 6 and 31 . 

Applicants traverse this rejection on the ground that neither Kupiec nor 
Bogrett disclose an examination of an information stream to recognize a 
presence of events in the information stream, wherein the events are derived 
from the stream based on one or more predetermined taxonomies. 
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The Examiner contends that Kupiec teaches the examining step, citing column 
14, lines 15-20, in support thereof. However, Kupiec's column 14 passage does 
not teach to recognize a presence of events in the information stream and to 
derive the events from the information stream based on one or more 
predetermined taxonomies. Kupiec contains no teaching of taxonomies. The 
Examiner contends that Kupiec's passages at column 5, lines 1-30, and column 
16, lines 30-36, support the taxonomy language of claims 3 and 6, respectively. 
However, neither of these passages mentions taxonomies. Accordingly, Kupiec 
does not teach taxonomies. Therefore, Kupiec does not teach the examination 
step of independent claims 1, 18, 24 and 33, as amended. 

Furthermore, Kupiec does not derive events from the information stream. 
Rather, Kupiec derives an utterance from spoken words and teaches to establish 
ambiguous interpretations of the utterance as a sequence of n words, which are 
re f erre d to as hypothesis by Kupiec. Kupiec proposes a system and method to 
disambiguate these hypotheses. Kupiec argues that if a sequence of words is a 
correct transcription of the input, the same sequence of words will occur also 
many times in a reference corpus. Towards this end, Kupiec's system retrieves 
documents (or parts of documents) that have the same sequence of words as a 
possible transcription of the utterance. The retrieval of the documents is based 
on applying a Boolean operator between the words of the hypothesis, and using 
a proximity and order operator in the retrieval engine. Clearly, some sequences 
will not be found in the reference corpus. The number of documents (and some 
other variations based on count) containing a sequence of words is used for 
scoring each hypothesis. Clearly, Kupiec's utterances are not events derived 
from the information stream. 

The Examiner admits that Kupiec does not teach (a) automatically 
generating database queries from the recognized events and (b) analyzing the 
results of the database queries so as to rank and select the results to be inserted 
into the information stream as collateral information, as recited in independent 
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claims 1 , 18, 24 and 33. The Examiner contends that Bogrett provides the 
deficiencies of Kupiec, citing Bogrett's column 1 1 , lines 50 and 51 , and column 7, 
lines 43-64. 

However, Bogrett's column 1 1 passage does not teach automatically 
generating database queries from events derived from the database based on 
one or more predetermined taxonomies. Bogrett teaches generating an SQL- 
string describing the user actions, based on manipulation of the graphical user 
interface. Bogrett's phrase "... automatically generates a database query. . . " 
actually refers to the automatic generation of a query based on user edits to a 
system generic query. Once the user has made the edits, Bogrett's step 294 
automatically generates the query. Thus, Bogrett merely teaches to 
automatically generate a database query based on user edits to a system 
generic query. Bogrett in no way teaches that the query be automatically 
generated based on events derived from the database based on one or more 
predetermined taxonomies, as recited in amended independent claims 1, 18, 24 
and 33. 

Furthermore, Bogrett's column 7 passage does not teach the step of 
analyzing results of the database query so as to rank and select the results to be 
inserted into the information stream as information collateral to the events. 
Bogrett's column 7 passage teaches about a system (the wizard) that helps 
users define their queries. The example given in which the wizard helps is that 
only tables (or rows/columns) for which the user has privileges are presented to 
the user for manipulation. There is no mention of rank or of collateral 
information. Bogrett only teaches with respect to relational databases. In such 
relational database systems, no ranking is provided. Bogrett in no way teaches 
analyzing results of the database query so as to rank and select the results to be 
inserted into the information stream as information collateral to the events, as 
recited in amended independent claims 1, 18, 24 and 33. 
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Accordingly, for the above reasons, Bogrett does not provide the admitted 
deficiencies of Kupiec. Therefore, the conclusion of obviousness is inapplicable 
to amended independent claims 1, 18, 24 and 33, and their dependent claims. 

With respect to claim 3, Kupiec's passage at column 5 lines 1-30, presents 
a glossary of terms that does not even contain the words "number of named 
entities" or "taxonomy path score", as recited in claim 3. Accordingly, Kupiec 
does not teach the specifics of the ranking based on a score from a free-text 
search, the number of named entities and a taxonomy path score, as recited in 
claim 3. 

With respect to claim 5, Bogrett's passage at column 7, lines 41-49, 
merely teaches the display relational data base models and user altering of these 
models. Bogrett does not teach the step of automatically extracting text from the 
information stream, as recited in claim 5. 

With respect to claim 6, Kupiec's passage at column 24, lines 51-55, 
teaches that the whole user input is used as input independent of how many 
words it comprises. Kupiec implicitly teaches that the user input is tokenized into 
words to determine co-occurrence of words. Kupiec does not teach about 
segmenting a textual input stream into sentences, as recited in claim 6. Kupiec's 
passage at column16, lines 30-36, does not teach about taxonomies. Rather, 
Kupiec teaches on imposing a proximity constraint on the query, which means 
that if two words in the query occur next to each other, the same two words occur 
next to each other in a relevant document. Relaxing the proximity constraint 
means that the two words can occur within n words of each other. Kupiec's 
column 16 passage does not identify topics that correspond to predetermined 
topic taxonomies, as recited in claim 6. 

With respect to claim 31, Bogrett does not teach that the queries are 
based on topics from a predetermined topic taxonomy. Rather Bogrett teaches 
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in column 1 1 , lines 50-51 , on how to construct a SQL query based on the 
manipulations by the users. The user manipulates relational database tables. 
Bogrett does not teach anything about taxonomies and free text search. Bogrett 
teaches about query models that include tables from a database, fields within 
these tables and links between the database tables. Accordingly, Bogrett does 
not teach that the queries are based on topics from a predetermined topic 
taxonomy, as recited in claim 31. 

The Office Action suggestion to use the combination of Kupiec and 
Bogrett is improperly based on the hindsight of Applicants' disclosure. Kupiec 
and Bogrett relate to entirely different systems. Kupiec teaches a semantic filter 
for speech recognition systems. In contrast, Bogrett teaches a relational 
database query system in which a generic query is manipulated by a user 
graphical interface. There is no motivation or suggestion for one of ordinary skill 
in the art to either select or combine Kupiec and Bogrett. 

Such hindsight reconstruction of the art cannot be the basis of a rejection 
under 35 U.S.C. 103. The prior art itself must suggest that modification or 
provide the reason or motivation for making such modification. In re Laskowski , 
871 F.2d 115, 117, 10 USPQ 2d 1397, 1398-1399 (CAFC, 1989). "The invention 
must be viewed not after the blueprint has been drawn by the inventor, but as it 
would have been perceived in the state of the art that existed at the time the 
invention was made." Sensonics Inc. v. Aerosonic Corp . 38 USPQ 2d 1551, 
1554 (CAFC, 1996), citing Interconnect Planning Corp. v. Feil , 774 F. 2d 1132, 
1138, 227 USPQ 543, 547 (CAFC, 1985). 

For the reasons set forth above, it is submitted that the rejection of claims 
1-3, 5, 6, 9, 18, 24, 31 and 33 under 35 U.S.C. 103(a) is inapplicable and should 
be withdrawn. 
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The Office Action rejects claims 4, 7, 8, 10, 13, 14, 20-22, 25-30 and 32 
under 35 U.S.C 103(a) as unpatentable over Kupiec in view of Bogrett and 
further in view of U.S Patent No. 5,835,667 to Wactlar et al., hereafter Wactlar. 

This rejection is inapplicable to claims 4, 7, 8, 10, 20-22, 25-30 and 32 for 
the reasons set forth above concerning the rejection of their parent claims 1,18 
and 24. 

With respect to claim 4, the Examiner admits that Kupiec and Bogrett do 
not teach that the database queries are automatically generated based on 
information corresponding to a list that identifies topics in text that is 
automatically extracted from the information stream, the topics corresponding to 
predetermined taxonomies. The Examiner contends that this deficiency is 
provided by Wactlar, citing column 15, lines 2-6. 

However, Wactlar does not provide the deficiency of Kupiec and Bogrett. 
Wactlar's column 15 passage describes a component of a video library retrieval 
system that allows extraction of video subsequences for reuse by the user. The 
selection of these subsequences is manual, and is aimed at supporting a video 
retrieval and editing system. Wactlar's column 15 passage in no way teaches 
automatically generating queries based on topics that correspond to 
predetermined topic taxonomies, as recited in claim 4. Therefore, Wactlar does 
not provide the admitted deficiency of Kupiec and Bogrett. 

The rejection of claims 13 and 14 is not understood, because their parent 
claim 12 was not rejected as obvious over the combination of Kupiec and Bogrett 
or the combination of Kupiec, Bogrett and Wactlar. Paragraph no. 8 of the Office 
Action contains no mention of claim 12 and no discussion of how claim 12 would 
be obvious over the combination of Kupiec and Bogrett or the combination of 
Kupiec, Bogrett and Wactlar. Therefore, it is submitted that the rejection of claims 
13 and 14 is erroneous and should be withdrawn. 
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With respect to claim 13, the Examiner contends that Kupiec's passage at 
column 12, lines 9-22, teaches that different scoring criteria can be used to score 
the hypotheses, i.e., the candidate transcriptions. Kupiec, as described above, 
uses search results to score automatic speech recognition transcription 
hypotheses (for the purpose of improving the accuracy of the speech transcript). 
This is different from and unrelated to the step of ranking the information 
extracted from the database for use by the multiplexing step. Kupiec does not 
teach ranking the extracted information for multiplexing of the ranked data into 
the information stream as recited in claim 13. 

With respect to claim 14, The Examiner contends that Kupiec teaches the 
step of ranking, citing the passage at 12, lines 1-6. However, Kupiec's column 
12 passage merely addresses scoring transcription hypotheses (i.e., candidate 
transcriptions of speech to text) and says nothing about named entities or a 
taxonomy path score that represents an amount of relatedness between a 
taxonomy-related information element identified in the text and a tree of the 
predetermined taxonomies, as recited in claim 14. 

With respect to claim 20, the Examiner contends that Wactler's passage at 
column 12, lines 57-59, teaches the unit for automatically extracting text, the unit 
for segmenting the text into sentences, and the unit for operating on the 
sentences to identify topics. However, Wactlar's column 12 passage says 
nothing about automatically identifying topics in the input stream based on a pre- 
defined topic taxonomy and generating queries at least in part on these topics. 
Wactlar is concerned with supporting queries explicitly and manually created by 
the end user. Wactler's column 12 passage addresses various ways in which 
video in the video library may be indexed to support these explicit end user 
queries. Wactlar says nothing about automatically generating queries to find 
collateral information. 
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With respect to claim 25, the Examiner contends that Wactlar's passage at 
column 6, lines 36-38, teaches the step of inserting the collateral information into 
the audio/video stream in real time or substantially real time. This contention is 
untenable. Wactlar's column 6 passage describes how arbitrary video may be 
stored in a video library system and associated with a specific final produced 
video and any raw video created during production of the final video. Selecting, 
associating, and storing this video in the video library system is a manual Xa&k, 
must be performed by a human, and teaches nothing about how to combine 
automatically discovered collateral information with the corresponding source 
video stream in substantially real-time. Accordingly, Wactlar's column 6 passage 
does not provide the step of inserting the collateral information into the 
audio/video stream in real time or substantially real time, as recited in claim 25. 

Regarding claim 26, the Examiner contends that Wactlar's passage at 
column 6, lines 41-48, teaches that the examining step includes a step of 
generating a speech transcript from at least the audio portion of the audio/visual 
stream and wherein the recognized events comprise speech topics. Wactlar's 
column 6 passage teaches how a known speech recognition system can be 
applied to generate a text transcript from the audio component of an audio/video 
stream. However, this teaching is strictly for purposes of indexing the 
audio/video stream based on the text transcript. This teaching has nothing to do 
with the collateral information, which is recited in claim 24 as being collateral to 
the events derived from the information stream. 

Regarding claim 28, Wactlar never considers meetings (i.e., discussions 
that take place at the meeting, audio, video, presentations, notes, other meeting 
artifacts), as the source of the input information stream, as recited in claim 28. 
The word "meeting" does not even appear in Wactlar. Wactlar's passage at 
column 1, lines 14-18 and 30-33, has nothing to do with meetings. 
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Regarding claim 29, as with claim 28, since Wactlar teaches nothing about 
meetings, Wactlar certainly teaches nothing about supporting an ongoing 
meeting by presenting collateral information to the meeting participants in real 
time. Wactlar Figure 4, step 42 concerns a user station for the video library, 
where the user can search the library and view video segments. This teaches 
nothing about supporting meeting participants with relevant collateral information 
automatically discovered based on an analysis of the meeting content. 

Regarding claim 30, since Wactlar does not address meetings at all, let 
alone the process we describe in claim 29, Wactlar could not teach how to 
archive the collateral information discovered to support a meeting. 

Regarding claim 32, Wactlar column 2, lines 44-53 refers to techniques for 
transcribing audio/video into text for purposes indexing and searching the video. 
In particular, Wactlar addresses the problem of finding a specific video segment 
about a given topic. This is unrelated to the technique claimed in claim 32 for 
identifying topics in an audio/video stream with respect to a given topic 
taxonomy, automatically generating queries based on the identified topics, and 
using the queries to search other related but completely separate databases for 
the purpose of finding collateral information relevant to the original video input 
stream. 

The Office Action rejects claims 12, 15, 16 and 17 under 35 U.S.C 103(a) 
as unpatentable over Wactlar in view of Bogrett. 

Independent claim 12 has been amended to recite that the text is 
analyzed to identify information elements based on one or more predetermined 
taxonomies. Claims 15-17 have also been amended for antecedent compatibility 
with amended claim 12. 
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The Examiner contends that Wactlar's passage at column 6, lines 42-47, 
teaches to analyze the text to identify information elements. However, this 
passage does not teach to identify the information elements based on one or 
more predetermined taxonomies. Therefore, Wactlar lacks the analyzing step of 
claim 12. 

The Examiner admits that Wactlar does not teach automatically generate 
queries from the information elements for searching at least one database, but 
contends that Bogrett does, citing Bogrett's column 1 1 , lines 50 and 51 . 

However, Bogrett's column 1 1 passage does not teach automatically 
generating database queries from information elements identified from the text 
based on one or more predetermined taxonomies. Bogrett teaches generating 
an SQL-string describing the user actions, based on manipulation of the 
graphical user interface. Bogrett's phrase "...automatically generates a database 
query..." actually refers to the automatic generation of a query based on user 
edits to a system generic query. Once the user has made the edits, Bogrett's 
step 294 automatically generates the query. Thus, Bogrett merely teaches to 
automatically generate a database query based on user edits to a system 
generic query. Bogrett in no way teaches that the query be automatically 
generated from information elements identified from the text based on one or 
more predetermined taxonomies, as recited in amended independent claim 12. 
Thus, Bogrett does not provide the admitted deficiency of Wactlar. 

Thus, the combination of Wactlar and Bogrett lacks the analyzing step and 
the automatically generating step of independent claim 12. 

With respect to claim 15, the Examiner contends that Wactlar teaches that 
the queries are generated based on a list of information elements identifying 
topics in the text being analyzed where the topics correspond to predetermined 
topic taxonomies. This contention is untenable. Wactlar's column 6 passage 

20 



teaches a (offline) system that can be used to create a digital library. In column 
14, lines 46-57, Wactlar teaches that a user can enter a query and how standard 
Natural Language Interpretation systems can summarize a query or identify 
named entities. Wactlar does not teach using a taxonomy to identify the 
information elements and to generate queries from the so identified information 
elements to search a database, as claimed in claim 15. 

With respect to claim 16, the Examiner contends that Wactlar teaches the 
step of segmenting text into sentences and the step of operating on the 
sentences to identify topics that correspond to topic taxonomies and wherein the 
step of automatically generating queries operates on identified topics, citing 
column 12, lines 57 and 58. This contention is untenable. Wactler's column 12 
passage teaches to search for video segments (called "paragraphs") using 
keywords. In this process, the end user is manually creating a query to search 
the video library and find matching video segments. This is completely different 
from the method recited in claim 16, where the system automatically segments 
the input text stream and associates it with topics for the purpose of automatically 
generating queries. These queries are then used to search other databases 
(text, video, whatever) to find new collateral information that is related to the input 
text stream. In Wactlar, the user manually creates the queries to search a video 
archive. Moreover, Wactlar does not teach how to use a topic taxonomy for this 
task. In the method of claim 16, the queries are automatically generated by the 
system using the topic taxonomy. 

With respect to claim 17, Wactlar (column 9, lines 41-56) teaches that 
named entities may be recognized and associated with a video segment (either 
from text, a transcript, or image elements) for the purposes of indexing the video 
segment using the named entities as keywords. However, Wactlar's column 9 
passage does not teach to identify the topics based on a topic taxonomy and the 
ranking of the database search results based at least on numbers of found 
named entities and on an amount of relatedness between a taxonomy-related 
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element identified in the text and a predetermined taxonomy tree, as claimed in 
claim 17. 

For the reasons set forth above, it is submitted that the rejection of claims 
4, 7, 8, 10, 13, 14, 20-22, 25-30 and 32 under 35 U.S.C. 103(a) is inapplicable 
and should be withdrawn. 

It is respectfully requested for the reasons set forth above that the 
rejections under 35 U.S.C. 103(a) be withdrawn, that claims 1-33 be allowed and 
that this application be passed to issue. 

For the reasons set forth above, it is submitted that this amendment 
places the application in condition for allowance. Accordingly, it is respectfully 
requested that this application be allowed and passed to issue. If this 
amendment is deemed to not place the application in condition for allowance, it is 
respectfully requested that it be entered for the purpose of appeal. 



Respectfully Submitted, 





Paul D. Greeley 
Reg. No. 31,019 
Attorney for Applicants 
Ohlandt, Greeley, Ruggiero & Perle, L.L.P. 
One Landmark Square, 10 th Floor 
Stamford, CT 06901-2682 
(203) 327-4500 
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